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ABSTRACT 

NASA at Marshall Space Flight Center (MSFC) and 
the U.S. Army at Redstone Arsenal were analyzed to 
determine whether they were successful in 
implementing their risk management program. Risk 
management implementation surveys were distributed 
to aid in this analysis. The scope is limited to NASA 
S&MA at MSFC, including applicable support 
contractors, and the U.S. Army Engineering 
Directorate, including applicable contractors, located at 
Redstone Arsenal. 

NASA has moderately higher risk management 
implementation survey scores than the Army. 
Accordingly, the implementation of the risk 
management program at NASA is considered good 
while only two of five of the survey categories 
indicated that the risk management implementation is 
good at the Army. 

INTRODUCTION 

The purpose of this project is to report the survey 
findings of the Risk Management Implementation at 
two government organizations. A survey developed by 
the author, entitled Risk Management , was used to 
solicit this data. The first organization is National 
Aeronautics and Space Administration (NASA) 
Marshall Space Flight Center (MSFC), and the second 
is the U.S. Army located at Redstone Arsenal. Both 
organizations work through matrix support provided to 
various projects and thus each project would dictate 
specific needs or requiremenls from the supporting 
team. The author will compare and contrast the two 
organizations’ implementation efforts. 

ORGANIZATIONS EVALU ATED 

This section introduces the two organizations to be 
evaluated. It contains a description of the risk 
management process utilized by both organizations. 
Additionally, it defines the risk management categories 
that will aid in evaluating the effectiveness of the risk 
management implementation. 


NASA. NASA was established in 1958 and has 
accomplished many great scientific and technological 
feats in air and space. NASA has also adapted 
technology for many uses by the private sector. This 
study focuses on a field installation of the National 
Aeronautics and Space Administration, the Marshall 
Space Flight Center, located in Huntsville, AL. MSFC 
was established in 1960 and named in honor of General 
George C. Marshall. General Marshall was the Army 
Chief of Staff during World War II, Secretary of State, 
and Nobel Prize Winner for his world-renowned 
"Marshall Plan." The survey focus at MSFC was the 
Safety and Mission Assurance (S&MA) team. A 
sample of approximately 100 contractors and civil 
servants are considered to have been involved in risk 
management implementation and thus were requested 
to participate in providing the survey results. 

NASA Risk Management. There are three 
requiiements documents tor risk management that 
NASA considers interdependent: 

• NPG 8705. XX (draft). Risk Management 
Procedures and Guidelines 

• NPG 7 120.5 A, NASA Program and Project 
Management Processes and Requirements 

• NPD 8700.1, NASA Policy for Safety and 
Mission Success 

Within Risk Management Procedures and 
Guidelines is the risk management plan and risk lists. 
Additionally, it contains the Program/project manager 
acts as the integrator of risk management. Ultimately, 
it provides additional information for applying risk 
management as required by NPG 7 120. 5A. 

The definition for risk management can be found 
in NPG 7 120.5 A: “an organized, systematic decision- 
making process that efficiently identifies risks, assesses 
or analyzes risks, and effectively reduces or eliminates 
risks to achieving the program goals.” Also found in 
NPG 7 120. 5 A is the NASA risk management process: 

• Identify risk issues and concerns 

• Evaluate (impact/severity, probability, 
timeframe), classify, and prioritize risks 
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• Decide what, if anything, should be done 
about risks 

• Monitor risk metrics and verify/validate 
mitigation actions 

• Decide to re-plan miligations, close risks, 
invoke contingency plans, or continue to track 
risks 

NASA’s policy can be found in the NASA Policy 
for Safety and Mission Success . The policy slates that 
using qualitative or quantitative risk assessment 
techniques will maximize the likelihood of mission 
success. Additional evidence of NASA s commitment 
and emphasis on risk management is in a NASA 
presentation (Dr. Michael Greenfield, 1998) titled Risk 
as a Resource. In his presentation, Dr. Greenfield 
states “effective project management depends on a 
thorough understanding of the concept of risk, the 
principles of risk management, and the establishment 
of a disciplined risk management process.” Dr. 
Greenfield also wrote a paper for NASA that addresses 
the need for risk to be managed differently such as the 
“knowledge-based” approach that NASA is moving to. 

NASA also conducts risk management training 
classes for civil servants as well as tor their 
contractors. The risk management class is presented by 
the NASA Safety Training Center. The class 
emphasizes that risk management and safety are 
correlated. The class teaches h»w a risk is an attribute 
of a hazard. Additionally, risk is an expression of the 
combined severity and probability of loss. NASA uses 
the convention for evaluating the severity of a risk for a 
hazard by working with the worst credible 
consequence. When considering probability, operating 
duration or number of trials/missions/operations is 
examined. To assess risk, both must be evaluated. A 
useful tool for assessing risk is a risk assessment 
matrix. A risk assessment matrix includes the 
relationship of probability against the severity of the 
consequence. Below is a simplified matrix. 
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Some, but not all, of the NASA risk management tools 
that are in place include: 


1. Fault Tree Analysis 

2. Failure Mode and Effect Analysis 


3. Probabilistic Risk Assessment 

4. Reliability Block Diagrams 

5. Risk Assessment Matrix 

Methods for establishing risk tolerance limits that are 
utilized by NASA include: 

• Formal analysis 

• Professional judgment 

• Bootstrapping 

Risk management roles and responsibilities are also a 
major factor in effective implementation. For NASA, 
performing risk management analysis is the 
responsibility of the line organizations or the staff 
specialists. However, the acceptance always falls on 
management. 

The U.S. Army. In 1941, congress approved funds for 
the Aimy to construct a chemical manufacturing and 
storage facility, Huntsville Arsenal, to supplement the 
production of the chemical manufacturing plant at 
Edgewood Arsenal. A facility, initially known as 
Redstone Ordnance Plant, was built adjacent to the 
chemical manufacturing installation. The plant was 
designated Redstone Arsenal in February 1943. 

The U.S. Army Aviation and Missile Command 
(AMCOM) Aviation & Missile Research, 
Development, and Engineering Center (AMRDEC) 
Aviation Engineering Directorate located at Redstone 
Arsenal in Huntsville, Alabama is the focus of this 
project. The Director of Aviation Engineering is the 
Airworthiness authority for Army developed aircraft 
and provides matrix support to their customers. 
Aviation Engineering direct customers are the Program 
Executive Officer Aviation Program/Project/Product 
Managers (PMs) and the U.S. Army Aviation and 
Missile Command (AMCOM) Defense Systems 
Acquisition PMs. Their ultimate customers are the 
Army aircraft crew, passengers, and maintainers that 
operate the Army aviation systems. The Engineering 
Directorate is made up ol approximately 660 
employees. The survey was distributed to about 100 
contractors and civil servants that were considered to 
have applicable knowledge of the risk management 
program implementation. 

The U.S. Army Risk Management. For the Army, 
risk is a way of measuring the potential that an event 
will result in a negative consequence. The Army has a 
risk management information system website that 
contains many useful tools and techniques utilized by 
the Army. Additionally lessons learned as well as 
safety information can be obtained from this site. 
Similar to the NASA philosophy, an Army Program 
Manager must consider the probability that an event 
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will occur and the consequences should that event 
occur when assessing risk. To ensure that DOD is 
acquiring optimum systems that meet all requirements, 
Program Managers must manage risk and assess cost, 
schedule, and performance. Once a risk is assessed, 
Program Managers must determine how best to handle 
it. Controlling risk, avoiding risk, assuming risk, and 
transferring risk are four strategies used. The four 
strategies can be use alone or in combination. 
Controlling the risk means lowe ing the chance that the 
event will occur. Avoiding the risk means changing 
the source that is subjecting the program to risk. 
Assuming the risk means planning for potential 
consequences. Transferring the risk means having 
someone else take accountability for the risk. 

Similar to NASA, the Army treats risk 
management as a process for identifying and 
controlling hazards to protect the force. Risk 
management is a proven accident-prevention process. 
According to BG James E. Simmons, director of Army 
Safety and commanding general of the U.S. Army 
Safety Center at Fort Rucker, AL, accident rates across 
the Army dropped following the adoption of risk 
management as the principle accident-prevention 
process. He also states that the Army’s most state-of- 
the-art safety weapon is risk management. Risk 
Management is the Army's principle risk-reduction 
process to protect the force. The Chief of Staff states 
the Army goal is “to make risk management a routine 
part of planning and executing operational missions". 
Another technique used by the Army is the five-step 
risk management process. According to BG Simmons, 
effectively applying the five-step risk management 
process will help do the right training safely and will 
also help execute operational missions safely. The 
Army’s Risk Management Card, which includes the 
five-step risk management process, follows. 



Risk Management 



The basic principles that provide a framework 
for implementing the risk management process are: 

• Integrating risk management into mission 
planning, preparation, and execution. 

• Making risk decisions at the appropriate 
level in the chain of command. 

• Accepting no unnecessary risk. 

Risk management integration strengthens risk 
management by embedding it in all the Army does, 
both on and off duty, as organizations and as 
individuals. Army risk management integration steps 
are: 

1. Identify risk management integration 

opportunities. 

2. Assess improvement opportunities. 

3. Develop integration procedures. 

4. Assist implementation of integration 

procedures. 

5. Measure and reassess the degree of 

integration and its results. 

Some, but not all, of the Army risk 

management tools that are in place include: 


1 . Safety Assessment Procedures 

2. Next Ground Accident Assessment for 
Individual 

3. Leader Training Support Package 

4. Soldier Training Support Package 
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5. Small Unit Risk Management Booklet 

6. Risk Management Cai d 

7. Protection (Safety) Readiness Checklist 
from Center for Army Lessons 

8. Risk Management Worksheet 

9. CECOM System Safety Lessons Learned 
Handbook 

10. Operation Risk Management Leader’s 
Guide 

In addition to risk management tools, below is a 
helpful listing of policy and doctrinal references related 
to Army risk management. 

• AR 70-1, Systems Acquisition Policy and 
Procedures, dtd 1997. 

• AR 385-16, System Safety Engineering and 
Management, dtd 3 May 90. 

• FM 100-14, Risk Management, dtd 23 April 
1998. 

• HQDA Letter 5-97-1, Risk Management 
Integration Responsibilities, dtd 1 May 97. 

• MIL-STD-882C, System Safety Program 
Requirements, 19 Jan c >3. 

• Center for Army Lessons Learned (CALL) 
Newsletter 99-5, "Risk Management for 
Brigades and Battalions", dtd Apr 99. 

• FM 101-5 Staff Organization and Operations, 
dtd 31 May 1997. 

Risk management roles and responsibilities are a 
little different for the Army than for NASA. 
Leadership at the appropriate level of authority making 
informed decisions to control hazards or accept risks is 
the Army standard for risk management. It is the 
responsibility and accountability of leaders to assess 
their operation as a total system and to ensure that 
planning, risk management decisions, and execution 
proactively identifies hazards, assesses the associated 
risks, and identifies control measures necessary to 
reduce the risks to the level commensurate with their 
commander's intent. 

The level of acceptance decision authority is 
determined by the degree ol risk. The risk issue must 
be elevated to the next higher command when 
resources to control a high risk are not available. This 
process promotes that a conscious and informed 
decision is made to commit the resources to control the 
hazards or accept the risk. 

EVALUATION. 

This section of the report evaluates the results from 
each organization individually. Each organization is 


measuied against criteria established in the distributed 
surveys. 

The surveys provide data based on five risk 
management categories. The risk management 
categories, including a demographics section, are. Risk 
Management Planning, Risk Identification, Qualitative 
and Quantitative Risk Analysis, Risk Response 
Planning, and Risk Monitoring and Control. By 
answering questions in each of these five categories, 
ranging from answers of strongly disagreeing to 
strongly agreeing, the respondents indicated whether or 
not their organization was successful in implementing 
risk management. A range of six to eleven questions in 
each of the five categories were answered and assigned 
a value based on the employee s level of agreement 
with five being considered the best score in terms of 
success. A score of three or below provided by the 
employee indicates a lack of success in this category of 
risk management implementation. A one was assigned 
for each answer of don't know or not applicable. An 
average of the questions was then calculated. 

NASA Risk Management Implementation Survey 
Results and Evaluation. NASA risk management 
implementation surveys were received from eighteen 
government and fifteen support contractors. Ot the 
thirty-three surveys, results were received from nine 
managers, one support staff, and twenty-three technical 
employees. Of those, 39.39% have worked at or 
supported NASA over seven years, 27.27% have 
worked there between one and three years, 18.18% 
have worked there between three and seven years, and 
15.15% have been there for less than one year. 

The survey results indicated that NASA was most 
successful in terms of Risk Management Planning, 
Risk Identification, and Risk Monitoring and Control 
with mean scores of 3.8. Qualitative And Quantitative 
Risk Analysis was next with a mean score of 3.7. Risk 
Response Planning barely ranked as a slightly positive 
score with a mean of 3.5. 

Although respondents indicated a successful score 
for Qualitative and Quantitative Risk Analysis, a 
weakness in the risk analysis process was in testing 
identified project assumptions against the stability ot 
the assumption and against the impact on the project if 
the assumption is false. 

Three weaknesses were identified in the Risk 
Response Planning category. One weakness was in 
changing the project plan to eliminate the risk and 
protecting the project objectives from the risk’s impact 
to avoid specific known risks. The other two 
weaknesses are in the risk response plan. The risk 
response plan does not allow for identification of 
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residual risks and/or secondaiy risks and does not 
allow for identification of contractual agreements. 


NASA Risk Management 



Risk Management Category 


Army Risk Management Implementation Survey 
Results and Evaluation. The Army risk management 
implementation surveys were received from twenty-six 
government and five support contractors. Of the thirty- 
one surveys, results were received from five managers 
and twenty-six technical employees. Of those, 54.84% 
have worked at or supported NASA over seven years, 
22.58% have worked there between one and three 
years, 16.13% have been there for less than one year, 
and 6.45% have worked there between three and seven 
years. 

The strongest area for the Army was Risk 
Identification with a mean score of 3.7 and Qualitative 
and Quantitative Risk Analysis was next following 
close with a mean score of 3.6. The other three 
categories indicate weak areas in risk management 
implementation for the Army. Risk Management 
Planning had a mean score of 3.3 while Risk Response 
Planning and Risk Monitoring and Control each had a 
mean score of 3.2. 

Although respondents indicated an unsuccessful 
score of 3.3 for Risk Management Planning, responses 
to individual questions indicated that the Army was 
strong in that it has a project charter or equivalent and 
is strong in decision making that influences planning. 

Qualitative and Quantitative Risk Analysis 
received an overall positive score, however, 
respondents indicated three weaknesses in this area. 
The respondents indicated that in the risk analysis 
process, identified project assumptions are not tested 
against the stability of the assumption and against the 
impact on the project if the assumption is false. A 
second weakness is that an overall risk ranking for the 
project is not provided by the risk analysis in order to 


assign personnel or other resources to projects with 
different risk rankings, to make a benefit-cost analysis 
decision about the project, and/or to support a 
recommendation for project cancellation. A third 

weakness is that risk analysis is not used to provide a 
prioritized list of quantified risks. 

Risk Response Planning received positive 

indications, although having an overall unsuccessful 
score, in the areas of the Army taking early action to 
mitigate risks and developing contingency plans in case 
the risk occurs. 

For Risk Monitoring and Control, although an 
overall unsuccessful score was indicated, a slightly 
positive score was achieved for using project 
performance and/or risk reports to monitor and control 
risks. 



Risk Management Category 


COMPARE AND CONTRAST ORGANIZATIONS 

This section will attempt to identify the similarities and 
differences in risk management implementation 
between the two organizations. 

Both are government organizations working 
undei a matrix structure that provides their risk 
management support. NASA had overall higher mean 
scores than the Army in each of the five categories. 
NAS A is considered successful in implementing their 
risk management program with an overall mean score 
of 3.77 while the Army is not deemed as having a 
successful program with an overall mean score of 3.41 . 
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Respondents indicated that the strongest risk 
management category for both organizations is Risk 
Identification. No obvious weaknesses for either 
organization were apparent in this category. Risk 
Management Planning ranked second strongest for 
NASA and ranked third for the Army. Risk 
Monitoring and Control ranked third for NASA and 
fourth for the Army, and Qualitative and Quantitative 
Risk Analysis ranked fourth for NASA and second for 
the Army in terms of success. Both organizations 
ranked the weakest in their Risk Response Planning 
category. 

An area for improvement for both organizations is 
in the Qualitative and Quantitative Risk Analysis 
category. Both are weak in testing identified project 
assumptions against the stability of the assumption and 
against the impact on the project if the assumption is 
false. Additional individual weaknesses, as well as 
noted strengths, are listed in the evaluation sections of 
this report. 

CONCLUSIONS 

This section contains the summary of the risk 
management implementation survey results for NASA 
and for the Army. 

The implementation of the risk management 
system at NASA is determined to be good. However, 
the following areas indicate a need for improvement: 

o Testing identified project assumptions against the 
stability of the assumption and against the impact 
on the project if the assumption is false. 


o To avoid specific known risks by changing the 
project plan to eliminate the risk and/or to protect 
the project objectives from its impact, 
o NASA should develop the risk response plan to 
allow for identification of residual risks and/or 
secondary risks. 

o NASA should develop the risk response plan to 
allow for identification of contractual agreements. 

The implementation of the risk management 
system at the Army is determined to be poor. Although 
survey results in each risk management category 
indicated an overall weakness^Rm the Army, the 
following areas indicate^Tl^ need for 

improvement: — 

o Testing identified project assumptions against the 
si ability of the assumption and against the impact 
on the project if the assumption is false, 
o Using risk analysis to provide an overall risk 
ranking for the project to assign personnel or other 
resources to projects with different risk rankings, 
to make a benefit-cost analysis decision about the 
project, and/or to support a recommendation for 
project cancellation. 

o Using risk analysis to provide a prioritized list of 
quantified risks. 

o Taking early action to mitigate the risk to reduce 
tlie probability and/or impact of a risk to below an 
acceptable threshold. 

o Developing a contingency plan in case the risk 
occurs once it is decided to accept the risk. 



Appendix A: Risk Management Implementation Assessment Survey Summary for NASA 
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RISK MANAGEMENT SURVEY 

SA = strongly agree; A= agree; D = disagree; SD = strongly disagree, NA = not 
applicable 

DEMOGRAPHIC 


1 Organization Type 

Government 

~ Support Contractor 

~2 Position in the organization 

management 

technical employee (l.e. engineer, designer, scientist) 


production employee 


support staff (l.e. clerical, human resource) 


3|Time in that position 


less than 1 year 


1 to 3 years 

3 to 7 years 


over seven years 


Number of employees at your specific site 


less than 25 


between 25 and 1 50 


between 1 50 and 500 


greater than 500 


RISK MANAGEMENT PLANNING 



SA A 


My organization has a project charter or equivalent that includes the business needs 
1 and project description at a level appropriate to the needs of the project. 


2 Risk management has not been used in my organization. 


3|My organization does not have predefined methods for qualitative risk analysis. 


My organization does not have predefined methods for quantitative risk analysis. 


My organization has predefined roles, responsibilities, and authority levels for 
5 decision-making that influence planning. 


6 Tolerances for risk are expressed in poli cy statements or revealed in actions. 

A template for my organization’s risk management plan exists and is adaptable to 
each project by the project manager or the risk management team. 


8 The risk management template is improv ed based on experience from each project. 
Meetings are conducted that are designed to adapt the risk management plan 

9 template to the current project. 

My organization’s risk management plan documents how risk identification, 
assessment, quantification, response planning, monitoring, and control will be 
10 structured and performed during the project life cycle. 



1 Process outputs are reviewed to identify pos sible risks. 

Risk categories are well defined and reflect common sources of risk for the industry 

2 or application area. 



















■ 


3 [Historical information on pricr projec ts is available for review by the project team. 

My organization performs structured documentation review(s) of one or more of the 
following: project plans and assumptions, prior project files, and other applicable 

information as an initial step by project teams. 

My organization utilizes one or more information gathering techniques in risk 

5 identification. 

My organization’s risk identification process provides adequate indications that a risk 

6 has occurred or is about to occur. 

A system is in place at my o ganization to use identified risks as inputs to other 
processes. 


QUALITATIVE AND QUANTITATIVE RISK ANALYSIS 


1 Risk probability and/or risk impact are risk analysis tools used by my organization. 


2 Probability / impact risk rating matrix is a risk analysis tool used by my organization. 
In my organization's risk analysis process, identified project assumptions are tested 
against the stability of the assumption and against the impact on the project if the 

3 assumption is false. 

My organization examines the extent of the understanding of a risk, the data 
available about the risk, the quality and integrity of the data, and the reliability of the 
data in order to evaluate the degree to which the data about risks are useful for risk 

management. 

Risk analysis is used to provide an overall risk ranking for the project which is used: 
to assign personnel or other resources to projects with different risk rankings, to 
make a benefit-cost analysis decision about the project, and/or to support a 

5 recommendation for project cancellation. 

Risks classified as high or moderate would be prime candidates for more analysis, 

6 including quantitative risk analysis, and for risk management action. 


My organization utilizes appropriate inputs for quantitative risk analysis 


As a part of the risk analysis process, my organization utilizes appropriate tools and 

8 techniques. 

Risk analysis is used by my organization to provide a prioritized list of quantified 

9 risks. 

Risk analysis is used by my organization to provide a probabilistic analysis of the 

10 project. 

Risk analysis is used by my organization to provide the probability of achieving the 

11 project cost and time object ves. 









RISK RESPONSE PLANNING 


To avoid specific known risks, my organization changes the project plan to eliminate 

1 the risk or condition and/or l o protect the project objectives from its impact. 

— To reduce the probability ard/or impact of a risk to below an acceptable threshold, 

2 my organization takes early action to mitigate the risk. 


If my organization decides to accept a risk, a contingency plan may be developed in 

3 case the risk occurs, or the project team may deal with the risk as it occurs. 

A risk response plan or equivalent exists and is written to the level of detail at which 

the actions will be taken. 

The risk response plan (or equivalent) allows for identification of residual risks and/or 
5 secondary risks. 

























The risk response plan (or equivalent) allows for identification of contractual 
6 agreements. 



RISK MONITORING AND CONTROL 


SA A 


SD NA 


Project performance and/or tisk reports are used to monitor and control risks. 


My organization implements risk identification, assessment, quantification and 
response planning for potential risks that surface as a result of measuring project 

2 performance. 

When required, my organization implements new risk analysis and response plans 

3 (or equivalent) as a result of scope changes. 

My organization utilizes appropriate tools and techniques for risk monitoring and 
control. 

Plans are updated as appropriate based on risk monitoring and control, workaround 

5 plans, corrective action, proj ect change requests, and/or risk response. 

My organization implements and maintains a risk database that is used in the risk 

6 management process. 

My organization updates the risk identification checklists (or equivalent) as 
appropriate based on risk monitoring and control. 


OPTIONAL QUESTIONS 


I consider the following tools and/or techniques to be greatly effective in my 
1 organizations risk management process: 


I do not consider the followirg tools and/or techniques to be greatly effective in my 
2 organizations risk managemen t process: 











